KANDA DATA

When we talk about regression analysis, we often think about parametric variables measured on at least an interval or ratio scale. But what if we want to analyze the effect of independent variables on a dependent variable that happens to be categorical in nature?

In this article, Kanda Data will walk you through this topic. I will show you that we can still use regression analysis even when the dependent variable is categorical—often referred to as a non-parametric variable.

Can We Use Regression If the Dependent Variable Is Categorical?

In quantitative research, we are usually accustomed to dealing with dependent variables measured on an interval or ratio scale. But what if the dependent variable we observe is a binary categorical variable measured on a nominal scale?

A binary categorical variable simply means it has only two categories. For example: Yes/No, Success/Failure, Healthy/Sick, or Buy/Not Buy. So there are only two possible outcomes for the variable we are observing. This binary categorical variable will then be treated as the dependent variable in our analysis.

When the dependent variable is binary, ordinary linear regression (Ordinary Least Squares/OLS) is no longer appropriate because linear regression assumes the dependent variable is continuous, unbounded, and normally distributed.

Moreover, when using OLS linear regression, we need to meet several assumptions to ensure we obtain the Best Linear Unbiased Estimator (BLUE). These assumptions include testing for residual normality, heteroscedasticity, multicollinearity, linearity, and autocorrelation (especially for time-series data).

For binary dependent variables, OLS linear regression simply doesn’t work. Instead, we use another method: Logistic Regression.

Why Can’t We Just Use Linear Regression?

If we insist on using linear regression for binary variables, the predictions might fall outside the 0–1 range (e.g., -0.3 or 1.5), even though probabilities should always stay between 0 and 1. Additionally, the assumption of residual normality is likely violated because binary data only take values 0 or 1. Residual variance is also likely to be non-homogeneous (heteroscedastic), which can lead to biased estimates.

For these reasons, logistic regression is preferred because it models the relationship between independent variables (X) and the probability of the binary dependent variable (Y) occurring.

The Basic Concept of Logistic Regression

Logistic regression differs from OLS linear regression in that it links the independent variables to the logit (log-odds) of the binary dependent variable.

Let’s use a simple example to make this clearer. Suppose we want to identify the factors influencing someone’s decision to buy a product online (Yes = 1, No = 0). The independent variables are Price (X1), Product Quality (X2), and Trust in the Store (X3).

After running logistic regression, the results show:

1. Product Quality (X2): A positive and significant effect (𝑝<0.05) with OR = 2.5. This means that increasing product quality raises the odds of buying online by 2.5 times.
2. Trust (X3): A positive and significant effect with OR = 1.8. Similarly, higher trust increases the odds of buying online by 1.8 times.

That’s all for today’s article from Kanda Data. Hopefully, this helps broaden your knowledge and gives you a clearer picture of logistic regression. Stay tuned for more educational content from Kanda Data!